Memory Hierarchy
What are the Design and Characteristics of Memory Hierarchy?
Memory Hierarchy, in Computer System Design, is an enhancement that helps in organising the memory so that it can actually minimise the access time. The development of the Memory Hierarchy occurred on a behaviour of a program known as locality of references. Here is a figure that demonstrates the various levels of memory hierarchy clearly:
This Hierarchy Design of Memory is divided into two main types. They are:
External or Secondary Memory
It consists of Magnetic Tape, Optical Disk, Magnetic Disk, i.e. it includes peripheral storage devices that are accessible by the system’s processor via I/O Module.
Internal Memory or Primary Memory
It consists of CPU registers, Cache Memory, and Main Memory. It is accessible directly by the processor.
Characteristics of Memory Hierarchy
One can infer these characteristics of a Memory Hierarchy Design from the figure given above:
1. Capacity
It refers to the total volume of data that a system’s memory can store. The capacity increases moving from the top to the bottom in the Memory Hierarchy.
2. Access Time
It refers to the time interval present between the request for read/write and the data availability. The access time increases as we move from the top to the bottom in the Memory Hierarchy.
3. Performance
When a computer system was designed earlier without the Memory Hierarchy Design, the gap in speed increased between the given CPU registers and the Main Memory due to a large difference in the system’s access time. It ultimately resulted in the system’s lower performance, and thus, enhancement was required. Such a kind of enhancement was introduced in the form of Memory Hierarchy Design, and because of this, the system’s performance increased. One of the primary ways to increase the performance of a system is minimising how much a memory hierarchy has to be done to manipulate data.
4. Cost per bit
The cost per bit increases as one moves from the bottom to the top in the Memory Hierarchy, i.e. External Memory is cheaper than Internal Memory.
Design of Memory Hierarchy
In computers, the memory hierarchy primarily includes the following:
1. Registers
The register is usually an SRAM or static RAM in the computer processor that is used to hold the data word that is typically 64 bits or 128 bits. A majority of the processors make use of a status word register and an accumulator. The accumulator is primarily used to store the data in the form of mathematical operations, and the status word register is primarily used for decision making.
2. Cache Memory
The cache basically holds a chunk of information that is used frequently from the main memory. We can also find cache memory in the processor. In case the processor has a single-core, it will rarely have multiple cache levels. The present multi-core processors would have three 2-levels for every individual core, and one of the levels is shared.
3. Main Memory
In a computer, the main memory is nothing but the CPU’s memory unit that communicates directly. It’s the primary storage unit of a computer system. The main memory is very fast and a very large memory that is used for storing the information throughout the computer’s operations. This type of memory is made up of ROM as well as RAM.
4. Magnetic Disks
In a computer, the magnetic disks are circular plates that’s fabricated with plastic or metal with a magnetised material. Two faces of a disk are frequently used, and many disks can be stacked on a single spindle by read/write heads that are obtainable on every plane. The disks in a computer jointly turn at high speed.
5. Magnetic Tape
Magnetic tape refers to a normal magnetic recording designed with a slender magnetizable overlay that covers an extended, thin strip of plastic film. It is used mainly to back up huge chunks of data. When a computer needs to access a strip, it will first mount it to access the information. Once the information is allowed, it will then be unmounted. The actual access time of a computer memory would be slower within a magnetic strip, and it will take a few minutes for us to access a strip.
The Computer memory hierarchy looks like a pyramid structure which is used to describe the differences among memory types. It separates the computer storage based on hierarchy.
Level 0: CPU registers
Level 1: Cache memory
Level 2: Main memory or primary memory
Level 3: Magnetic disks or secondary memory
Level 4: Optical disks or magnetic types or tertiary Memory
In Memory Hierarchy the cost of memory, capacity is inversely proportional to speed. Here the devices are arranged in a manner Fast to slow, that is form register to Tertiary memory.
Let us discuss each level in detail:
Level-0 − Registers
The registers are present inside the CPU. As they are present inside the CPU, they have least access time. Registers are most expensive and smallest in size generally in kilobytes. They are implemented by using Flip-Flops.
Level-1 − Cache
Cache memory is used to store the segments of a program that are frequently accessed by the processor. It is expensive and smaller in size generally in Megabytes and is implemented by using static RAM.
Level-2 − Primary or Main Memory
It directly communicates with the CPU and with auxiliary memory devices through an I/O processor. Main memory is less expensive than cache memory and larger in size generally in Gigabytes. This memory is implemented by using dynamic RAM.
Level-3 − Secondary storage
Secondary storage devices like Magnetic Disk are present at level 3. They are used as backup storage. They are cheaper than main memory and larger in size generally in a few TB.
Level-4 − Tertiary storage
Tertiary storage devices like magnetic tape are present at level 4. They are used to store removable files and are the cheapest and largest in size (1-20 TB).
Let us see the memory levels in terms of size, access time, bandwidth.
| Level | Register | Cache | Primary memory | Secondary memory |
| Bandwidth | 4k to 32k MB/sec | 800 to 5k MB/sec | 400 to 2k MB/sec | 4 to 32 MB/sec |
| Size | Less than 1KB | Less than 4MB | Less than 2 GB | Greater than 2 GB |
| Access time | 2 to 5nsec | 3 to 10 nsec | 80 to 400 nsec | 5ms |
| Managed by | Compiler | Hardware | Operating system | OS or user |
Why memory Hierarchy is used in systems?
Memory hierarchy is arranging different kinds of storage present on a computing device based on speed of access. At the very top, the highest performing storage is CPU registers which are the fastest to read and write to. Next is cache memory followed by conventional DRAM memory, followed by disk storage with different levels of performance including SSD, optical and magnetic disk drives.
To bridge the processor memory performance gap, hardware designers are increasingly relying on memory at the top of the memory hierarchy to close / reduce the performance gap. This is done through increasingly larger cache hierarchies (which can be accessed by processors much faster), reducing the dependency on main memory which is slower.